Necessary and Sufficient Conditions for Recurrence and Transience of Markov Chains, in Terms of Inequalities

For an aperiodic, irreducible Markov chain with the non-negative integers as state space it is shown that the existence of a solution to E-,op,ijy, y,; i N > 0 in which y, -* 0 is necessary and sufficient for recurrence, and the existence of a bounded solution to the same inequalities, with yk < y, , yN-1 for some k N, is necessary and sufficient for transience. RECURRENCE; TRANSIENCE Consider a Markov chain with the non-negative integers as its state space and transition matrix P = (p,), and for simplicity assume that the chain is irreducible and aperiodic. Theorem 1. A necessary and sufficient condition that the Markov chain be recurrent is that there exists a solution to the system of inequalities (1) E pijyj, yi; i N, for some integer N > O, such that lim y, = c. j =( Sufficiency of the condition of Theorem 1 is well known. For N = 1 this is Theorem 5 in Foster (1953). Foster states that 'the condition would appear necessary only under certain additional assumptions'. For N 1 the sufficiency is proved in Pakes (1969), Theorem 3. For the sake of completeness, and in its own interest, we include a different proof of the sufficiency. Proof of sufficiency. Let y be a solution of (1) such that limy, = c. By possibly adding a constant to all components of y we may without loss of generality assume y, i 0. Define y * by y * = y, for i N, y * for i < N and let P* be obtained from P by setting p * = p,, for i > N, and p *i = 1 for i < N. Then P* and y* satisfy =(,p*y * y y * i 0. Let Zo = N and let Z,; n 1, be the state at time n of the Markov chain governed by P*. Note that the original chain Received 14 October 1977; revision received 26 January 1978.